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Abstract 



Community colleges must often analyze and report rates for outcomes, such as transfers to four- 
year schools. A single, summary rate for a college’s performance may be an invalid measure of 
its achievement in the transfer goal (or any other selected goal, for that matter) if the summary 
rate ignores the real differences in enrollment composition at different institutions. 

This paper presents a method for adjusting a college’s outcome rate to compensate for 
differences in enrollment composition. This method, known as standardization of rates, has two 
methods of calculation, the direct method and the indirect method. 

For didactic purposes, both methods are demonstrated here on data for transfers and age 
composition at one California community college. Some general guidelines for the use of 
standardization are covered. 
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Introduction 



College administrators and college researchers have a growing need to analyze the process and 
outcome data for their institutions. Two major purposes of such analysis are: 

• To advance our understanding of causal forces and 

• To implement accountability programs. 

These two purposes share a common thread. They must rely upon valid comparisons of data in 
order to draw conclusions about individual institutions. 

California’s community colleges embody a very diverse set of local environments. The diversity 
of environments presents a challenge to the analyst who must compare a specific institution to 
others in the system. If the analyst does not control for “confounding” variables in the 
environment, an analysis may draw invalid conclusions about a specific, hypothesized cause- 
and-effect relationship of concern. 

The control of confounding variables matters a great deal for causal analysis. If the analyst 
concludes that no relationship between an intervention and an outcome exists, when the 
environmental variables have simply hidden the relationship, then a college may unfortunately 
abandon an intervention that really would have worked. 

The control of confounding variables matters a great deal for accountability programs as well. If 
a college’s performance evaluation uses an inequitable comparison, comparing unlike colleges as 
if they were truly equivalent in environments, then again one may reach an invalid conclusion 
about how well that college’s administration has done. With an invalid comparison of college 
outcomes, a college may receive either undeserved blame or undeserved credit. Neither result is 
desirable. 

Over the decades, analysts have used a number of statistical methods to adjust data for the 
confounding bias that demographic characteristics have upon outcome measures. The method 
known as standardization is one of these tools to produce data that permit a valid comparison 
between specific subunits of a system, such as colleges. This method standardizes college- 
specific data to counteract the confounding of “extraneous” variables in a causal process under 
review. The discussed method has had a long tradition of use in the fields of health research and 
demography. For researchers who have worked in these fields, the method is a fairly common 
tool in their analysis and reporting. 

Note that standardization is unnecessary if a college can report outcome rates for specific strata 
in its enrollment population. This level of detail enables a college to state precisely how a 
difference in enrollment composition relates to a difference in outcome rate. Unfortunately, 
many users of such outcome data prefer the use of summary measures, rather than arrays of 
tables, so that they can make simple comparisons across many different units, or levels, of 
service delivery. 
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The Method 



In general, standardization works by producing an outcome rate that we assume would occur if a 
college had the same enrollment composition as the group with which we compare it. The group 
with which we want to compare a college is called the standard population . 

For community colleges, we might use any of the following as the standard population, 
depending upon the goal of our comparison. 

1 . State-wide total enrollment at community colleges 

2. Another college 

3. The total enrollment for a college’s district 

4. A college’s enrollment at a chosen time period 

The standard populations given in #1, #2, and #3 above specify differing levels of aggregation 
for college data, but the data are for a common time period in each case. For example, we would 
use the statewide total enrollment (#1) as our standard population if we wanted to compare 
outcomes from all of the state’s colleges. Or a district administrator might use the district’s total 
enrollment as a standard population if the administrator were to want a comparison between 
colleges within the district. 

The standard population in #4 differs from the others because it has a unique, but valuable, use. 
If we want to see how, or if, a college’s outcome rate has changed over the years, then we would 
choose the college’s enrollment composition for a given year as the standard population and 
adjust the rates for all of the other years of outcome data. This can be an important use of 
standardization, especially where (a) a college wants to compare a rate over many years or (b) 
meaningful change in enrollment composition at the college has occurred between the time 
periods used in the comparison. However, for brevity’s sake, we will focus on the statewide 
total enrollment as our standard population in the following demonstration of standardization. 

In standardization, we call the raw, unstandardized rate the crude rate . The outcome rate that we 
have standardized is called the adjusted rate (or the standardized rate) . If we were to standardize 
a rate for the variable gender, we would call the resulting rate a gender-adjusted rate . 
(Historically, researchers have used the phrase “sex-adjusted rate.”) 

In an ideal situation, we would have complete outcome data for each stratum (or grouping or 
level) of the adjustment variable that we suspect will make a difference in the outcome rate. For 
instance, in a comparison of a college’s transfer rate with the statewide transfer rate, we would 
need the following data to adjust for a difference in age composition: 

• The college’s transfer rate (the crude rate) for each stratum of the age variable and 

• The proportion of the state’s enrollment that is in each stratum of the age variable. 

This amount of data lets us use the method called direct standardization . If we lack the college’s 
outcome data for the strata under consideration, we can still produce an adjusted rate by using 
the method called indirect standardization . This second method is less preferred because it tends 
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to be slightly less precise than the direct method. To show how we would calculate standardized 
rates, with either method, a demonstration using real data is our next section. 

Demonstration 



For didactic purposes, we will use transfer data for Mission College in this demonstration. 
Figure 1 displays the two major steps in direct standardization. It is preferable to have count data 
from which to calculate the proportions and rates at both the college level and the statewide 
level. The volume of cases in each stratum will help you decide if the standardization is usable. 
If counts are too small for certain strata (or if zeroes appear), you may need to consider 
collapsing (that is, combining) similar strata or skipping standardization as a method. Without 
counts for the strata, we cannot collapse strata as described above because we would not know 
what weight to give each stratum’s transfer rate. 

In this demonstration, we dichotomized the age variable into (a) students who were under 25 
years of age at the start of the cohort period or into (b) students who were 25 or more in age at 
the start of the cohort period. This is a simplification that works in this situation. In health 
research and demography, the age variable usually has strata intervals of ten years in length. 
Ten-year intervals would not work here because we would have many strata with few or no cases 
in them. 

The calculations were done in Microsoft Excel but analysts may do them with hand calculators 
almost as easily. Note also that the rate is in a format of cases per thousand students. This 
format is a convention in the standardization practice to eliminate the need, and inconvenience, 
of reading a set of long decimal numbers in the tables. 

Direct standardization in this example indicates that Mission College’s transfer rate closely 
matches the statewide rate. Thus, when we adjust for age, Mission College’s transfer 
performance is better than we might initially believe, given the lower figure represented by the 
commonly used crude rate. This example also indicates that age composition is a meaningful 
variable in estimating transfer performance at Mission College. If the age-adjusted rate had been 
very close (or the same as) to the crude rate, we would tend to believe that age composition is not 
a meaningful variable in transfer performance at this college. 
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Figure 1. Example of Direct Standardization 



Example of Calcuation to Adjust Transfer Rate for Age Composition* 

Step 1: Assemble Needed Data* for "Crude Rates" 



Mission College 





count 


proportion 


Under 25 age 


1252 


0.4791 


age 25 or more 


1361 


0.5209 


total 


2613 







transfers 


tran.rate 


Under 25 age 


138 


0.1102 


age 25 or more 


18 


0.0132 


total 


156 


0.0597 



"Standard Population" of State's CC's 



count proportion 



Under 25 age 


224836 


0.6521 


age 25 or more 


1 1 9966 


0.3479 


total 


344802 







transfers 


tran.rate 


Under 25 age 


24775 


0.1102 


age 25 or more 


2686 


0.0224 


total 


27461 


0.0796 



Step 2: Apply State Proportion to College's Data to Find the "Adjusted Rate." 



transfer rate adjusted for age composition of a college: 





A: raw transfer 


B: proportion in 






rate at a 


"standard 


A x B (or 




college 


population" 


adjusted rate) 


Under 25 age 


0.110224 


0.6521 


0.071874 


age 25 or more 


0.013226 


0.3479 


0.004602 



sum of AxB for age groups: 



0.076475 or in terms of rate per 1,000 students: 



76.48 



state-level rate: 0.079643 or in terms of rate per 1,000 students: 79.64 

college’s crude rate: 0.059701 or in terms of rate per 1 ,000 students: 59.70 
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If we had no transfer rate data for the age strata at a college, we could resort to indirect 
standardization. Figure 2 on the next page shows how to calculate an age-adjusted rate with this 
alternative method. To simulate the absence of the data used in the direct standardization 
calculation (Figure 1), “unknown” appears in the table of data for Step 1 of Figure 2. 

The big difference in Figure 2 is the use of an “expected transfer count” as a substitute for the 
missing data. Note that indirect standardization still requires stratum-level counts for the age 
variable at the college. Without the stratum-level counts, we could not do an indirect 
standardization to adjust for age composition. 

In this example, the standardized rate with the indirect method still shows a higher outcome rate 
than the crude rate for Mission College. However, it is slightly lower than the rate found with 
the direct method. We can summarize the results of this demonstration in Table 1 . 

The ratio of the adjusted rate to the crude rate gives us a rough measure of the effect of adjusting 
for age. In this case, the effect with direct standardization is 76.475 divided by 59.701, or 1.28. 
Because this ratio is greater than one, the adjustment has an increasing effect. If it had been less 
than one, the adjustment would be a decreasing effect. This ratio can be used to evaluate the 
relative effect that different variables have in the standardization method. 

The ratio can also indicate, in a gross manner, the relative difference between enrollment 
composition variables in their inferred effect on an outcome rate. But this inference is a tenuous 
one; multivariate statistical analysis is much more suitable to address the quantification of 
relative effect upon an outcome rate. 



Table 1. Summary of the Results of the Demonstration 



1 . Mission's rate by indirect method: 


0.073764 or in terms of rate per 1 ,000 students: 


73.76435 


2. Mission's rate by direct method: 


0.076475 or in terms of rate per 1,000 students: 


76.47537 


3. state-level rate: 


0.079643 or in terms of rate per 1 ,000 students: 


79.64281 


4. Mission's crude rate: 


0.059701 or in terms of rate per 1 ,000 students: 


59.70149 
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Step 1 : Assemble Needed Data for "Crude Rates' 



Mission College 





count 


proportion 


Under 25 age 


1252 


0.4791 


age 25 or more 


1361 


0.5209 


total 


2613 







transfers 


tran.rate 


Under 25 age 


unknown 


unknown 


age 25 or more 


unknown 


unknown 


total 


156 


0.0597 



"Standard Population" of State's CC's 





count 


proportion 


Under 25 age 


224836 


0.6521 


age 25 or more 


119966 


0.3479 


total 


344802 







transfers 


tran.rate 


Under 25 age 


24775 


0.1102 


age 25 or more 


2686 


0.0224 


total 


27461 


0.0796 



Step 2: Apply State Proportion to College's Data to Find the "Expected Transfer Count." 





fie standard 


B: count of 






population's 


students per 


A x B (or 




transfer rate 


group at a 


"expected 




per group 


college 


transfers’) 


Under 25 age 


0.1102 


1252 


137.9597 


age 25 or more 


0.0224 


1361 


30.47235 



sum of AxBfor age groups: 168.432 



Step 3: Use Expected Transfer Count to Calculate College's Adjusted Transfer Rate 



College's Adjusted Rate = (total transfers at a college/ coNegeVto^ transfers) x state's rate 

168.432 ) X 0.0796 



/ 

= ( 156 / 

= 0.073764 

rate with indirect standardization: 0.073764 or in terms of rate per 1 ,000 students: 73.76435 



*Dat a as found by 2/29/2000 query on the 1994 Expanded Student-Right-To-Know database 



Figure 2: Example of Indirect Standardization 
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Guidelines 



Someone who wants to get a complete grasp of standardization should consider the works listed 
in the References section at the end of this paper. In the meantime, there are some relatively 
simple guidelines to note. Below are things to remember in using standardization, whether the 
outcome is transfer rate or some other event. 

A. Standardization of rates can help you in two common types of comparisons. 

• A comparison of a college’s rate to the rate(s) of one or more colleges in one 
time period. 

• A comparison of a college’s rate in different time periods. 

B. To help the first-time user of standardization here are the basic steps to follow. 

1 . Select the outcome rate to standardize. 

2. Select the variable for which you need to adjust. 

3. Select a “standard population.” 

4. Check the consistency of the data for the different levels in the analysis. (Is 
data captured in the same manner for each college and for each group in the 
adjustment variable?) 

5. Select an appropriate grouping for the adjustment variable. 

6. Assemble data in the chosen format. 

7. Calculate the adjusted rate. 

8. Compare the adjusted rate to the unadjusted (or “crude”) rate to see if the 
adjusted rate differs in a meaningful way. 

9. If you use the adjusted rate, document your calculation of it. 

10. When you use the adjusted rate, label it appropriately. 



10 
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Figure 3 summarizes general considerations in the use of direct standardization. Figure 4 
summarizes additional considerations in the use of indirect standardization. 

Figure 3. Some points to remember about direct standardization. 



1 . The adjusting variable should be one that has relevance to the causal process measured. 

2. If you adjust for a "continuous" variable, the grouping interval can make a difference. 

3. The standard population may be any one you choose; the state total is used here because I 

attempt to compare colleges against the system as a whole. 

4. This method becomes awkward when you need to adjust for more than a couple of variables. 

5. If the data you adjust are sample data, then calculate the confidence intervals. 

6. The method can adjust for compositional differences of the same data elements from differing 

time spans, not just across geographically differing reporting units. 



Figure 4. Additional points for indirect standardization. 



1 . Useful if stratum-specific transfer rates are unavailable. 

2. Useful if stratum-specific transfer rates are very small, hence unreliable. 

3. Less preferred to direct standardization because of slightly less accuracy. 
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Conclusion 



Not every college will benefit by standardizing one or more of its outcome rates. A college will 
benefit from the method if (a) it has an enrollment composition that diverges from the statewide 
composition and (b) the college’s outcome rates vary meaningfully between the strata of the 
adjustment variable. Both (a) and (b) should exist to make standardization worthwhile. 

Second, the finding of a large difference between a standardized rate and a crude rate is limited 
evidence of a relationship between the adjustment variable (also called the “controlled” variable) 
and an outcome measure. The adjustment variable may only have a statistical correlation with 
the outcome measure. As we should all remember, statistical correlation between two variables 
is necessary, but not sufficient, evidence of a causal relationship between these variables. 

In closing, remember that even with (a) and (b) in effect, standardization will probably give only 
a partial remedy, if any, to the problem of invalid summary outcome measures. Validity is rarely 
an “all-or-nothing” issue; standardization to adjust for a relevant variable will make a summary 
measure relatively more valid than no adjustment at all. If the standardized rate still contains the 
effect of one or more major confounding variables, then someone will need to do more analysis, 
perhaps with more sophisticated methods, to produce a more valid outcome measure. 
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